Exploring Public Sentiment on Twitter: An NLP Approach

Part 1: Setting Up

Task 1: Import the Libraries

Part 2: Data Collection and Preprocessing

Task 2: Load the Dataset and Have a First Look

Load the CSV into a variable named df_twitter
Display the DataFrame along with the row count
Show the tweet count for the top 10 countries
Plot the top 20 users who post the most

Task 3: Basic Text Preprocessing

Remove unnecessary columns for the analysis
Check for missing values and handle them
Convert date column to datetime object and extract features
Apply basic_clean_text() function to text column

Task 4: Implement Advanced Text Preprocessing

Apply advanced_text_preprocessing() function to text column

Part 3: Sentiment Analysis

Task 5: Perform Sentiment Analysis with Vader Library

Perform sentiment analysis with vader library
Display a random sample of 10 tweets with their sentiment scores

Task 6: Classify the Tweets into Positive, Neutral and Negative

Classify the tweets into categories of positive, negative, or neutral sentiment

Part 4: Trend Analysis and Visualization

Task 7: Display the Evolution of Sentiment Over Time

Extract the top 3 countries to be used as a filter
Filter the DataFrame
Create sentiment over time by country based on groupby
Plot the top three countries sentiments over time

Task 8: Use Wordcloud to Visualize Words Used in Sentiments

Define the create_word_cloud() function
Creating word cloud for positive sentiment
Creating word cloud for negative sentiment
Creating word cloud for neutral sentiment

Task 9: Display the Sentiment on a Geographical Heatmap

Create mapping from positive, neutral and negative to an numerical value for visualization purpose
Visualize a geographical heatmap of all sentiment on the map

Part 5: Topic Modeling

Task 10: Train the LDA (Gensim) Model

Preprocess the text
Create a dictionary
Filter extremes
Create a corpus
Instantiate an LDA model

Task 11: Evaluate the Model

Compute the perplexity and coherence score

Task 12: Classify Twitter Tweets into Topics

Topic classification function
Classify all tweets
Examine topic distribution

Part 6: Interpretation of Results

Task 13: Identify Relationships between Sentiment and Topic

Grouping and aggregating data by topic and sentiment
Calculating proportions
Visualizing the results

Task 14: Interpret the Topic Modeling Results

Creating topic distance visualization

Task 15: Compile your Findings into a Final Report with NBConvert

Execute the command within the provided notebook cell

CONGRATULATIONS